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Abstract. We consider the distribution of the length of the longest 
subsequence avoiding an arbitrary pattern, tt, in a random permu- 
tation of length n. The well-studied case of a longest increasing 
subsequence corresponds to tt — 21. We show that there is some 
constant such that as n — + oo the mean value of this length 
is asymptotic to 2^/c n n and that the distribution of the length is 
tightly concentrated around its mean. We observe some apparent 
connections between c n and the Stanley- Wilf limit of the class of 
permutations avoiding the pattern tt. 



The aim of this paper is to generalize certain results concerning the 
longest increasing subsequences of permutations to longest subsequences 
avoiding some pattern or patterns. The former area is quite well known, 
while the latter, though an active field of current combinatorial re- 
search, is considerably less so. Thus before surveying the known results 
(or at least the ones relevant to this paper) concerning longest increas- 
ing subsequences, we introduce the fundamental definitions concerning 
pattern avoidance in permutations. 

Let o = o\o<i ■ ■ ■ Ok and tt = 7t 1 7T2 • • • 7T n be permutations written as 
sequences. Then a occurs as a pattern in tt (or a is involved in tt) if 
some length k subsequence of tt has the same relative ordering as a. In 
other words, for some 1 < i\ < • • • < ik ^ n and all 1 < s, t < k: 



For example, a permutation ir contains an increasing subsequence of 
length k if and only if 123 • • • k occurs as a pattern in tt. We say that 7r 
avoids a if a is not involved in tt. A pattern avoidance class, Av(B) is 
the set of all permutations that avoid each of the permutations in the 
set B. The set Av(B) is infinite unless B contains both an increasing 
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and a decreasing permutation (in which case it is finite by the Erdos- 
Szekeres theorem) and if B is non empty then Av(B) n S n is a proper 
subset of «S n for all n greater than or equal to the length of the shortest 
element of B. 

More generally, given a finite sequence a of distinct values from any 
linearly ordered set, we define its pattern to be that permutation whose 
elements are in the same relative order as those of a. Then involvement 
can be defined for such sequences by reference to their patterns (or 
directly as above). 

Recently Marcus and Tardos [TH] proved the Stanley- Wilf conjecture 
about pattern avoidance classes. Specifically, 

Theorem (Marcus- Tardos Stanley- Wilf). For any proper pattern avoid- 
ance class A there is a real number such that: 

limsup \ S n fl A\ l ^ n = sa- 

So, not only is Av(£>) fl S n a proper subset of S n for sufficiently large 
n, it is in fact small relative to S n , being of merely exponential size in 
n. We call s^ the Stanley- Wilf limit of the class A. 

Consider an arbitrary pattern avoidance class A. Given a permutation 
n define the longest A subsequences of n or L^4S(7r) to be the set of 
those subsequences of 7r of maximum length, subject to the condition 
that their patterns belong to A. Also define L^(tt) to be the length 
of any sequence in LAS(tt). Let X = Av(21) be the class of increasing 
permutations (note that we should really write Av({21}) but we will 
frequently omit such braces for explicit sets of avoided patterns). Then 
LXS(7r) is simply the set of longest increasing subsequences of it. 

Apparently Ulam [21] was the first to ask the question: 

What can be said about the distribution of values of 
Lj(n n ) when H n is a random variable whose value is a 
permutation n chosen uniformly at random from among 
the elements of S n ? 

We intend to address the generalization of this problem to the random 
variable L^(U n ) defined in a similar fashion. The history of the analysis 
of Ulam's problem is well documented in |2J. We repeat here a few 
details relevant to our investigations of the more general problem. 

For convenience let L n = Lj(H n ). Ulam conjectured that for some 
constant c 

,. E L n 
hm — — = c. 

n-»oo Jfl 



LONGEST PATTERN AVOIDING SUBSEQUENCES 



3 



This conjecture was proven by Hammersley ^2] who showed also that 
n~ l l 2 L n — > c in probability, and who conjectured that c = 2. This 
further conjecture was proven in part by Logan and Shepp [T^] and 
simultaneously in whole by Kerov and Versik [22] . 

Frieze jH] and Bollobas and Brightwell jE] used martingale methods 
to establish tight concentration of L n about its mean. Subsequently, 
Baik, Deift and Johansson [5 j obtained complete asymptotic informa- 
tion about the distribution of L n . 

The evaluation of Lj{ir) for a specific permutation 7r is straightforward, 
owing to the fact that it is the length of the first row of the Young 
tableau obtained from 7r using the Schensted algorithm • For an ar- 
bitrary permutation tt this row is easily constructed in worst case time 
0(|7r| logLj(7r)). Furthermore, the Robinson-Schensted-Knuth corre- 
spondence allows exact computation of the distribution of L n for mod- 
erate values of n as was done by Baer and Brock |3] and to larger values 
by Odlyzko and Rains . Monte Carlo simulation of the distribution 
of L n for much larger values of n carried out in the early 1990 's but 
reported in led to new conjectures concerning the asymptotic vari- 
ance of the distribution, conjectures subsequently proved correct in j^j. 

Our main theorems are analogs of the results of Hammersley and Frieze 
for the more general case of L^(U n ). For the first of these we need to 
impose a mild additional restriction on A. The proofs for the general 
case are then essentially identical to the originals. We will illustrate an 
apparent connection between the limit of the expectation of the longest 
A subsequence and the constant together with some simulation 
results for classes where we have been unable to compute the limiting 
ratio exactly. 



2. Longest A subsequences in random permutations 

Let a proper pattern avoidance class A = Av(-B) be given. Choose 
some constant s such that for all sufficiently large n, 

|.AnS n | < s n . 

The existence of such a constant follows from the Marcus- Tardos Stanley- 
Wilf Theorem. 

As in the arguments of it is convenient to work with a different ran- 
dom variable than L^(H n ). Let X = (Xi, X 2 , . . . , X n ) be a sequence of 
independent uniform [0, 1] random variables. Define the random vari- 
able L_a(X.) to be the length of the longest subsequence of X whose 
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pattern belongs to A. This random variable depends only on the pat- 
tern of the sequence X and these patterns are uniformly distributed 
over <S n . 

The proof in jU] begins with a crude probability inequality for L n . We 
prove essentially the same inequality in exactly the same way, allowing 
only for the growth rate of the class A (bounded by s n ) as opposed to 
that of X (bounded by 1). 

Lemma 1. 

Pr (La(X) > 2ev^) < e" 26 ^. 

Proof. Let n = \2e^/sn\ , and let a denote the number of subsequences 
of X of length n whose pattern belongs to A. Then: 

Pr (L4(X) > n ) < 

< 

< 

< 

□ 

Define the direct sum a © f3 of two permutations a and (3 to be that 
permutation a/3' where the pattern of f3' is the same as that of j3 and 
each element of f3' is larger than all the elements of a. Similarly the 
direct difference a (3 is defined to be that permutation a! (3 where 
the pattern of a 1 is the same as that of a and each element of a 1 is 
larger than all the elements of (3. A pattern class A is said to be sum- 
closed (respectively difference-closed) if for all a, (3 G A, a © (3 G A 
(respectively a (3 G A). 

Much of the investigation of pattern avoidance classes has focused on 
the principal classes, those of the form Av(7r) for some single permuta- 
tion 7r of length at least two. Any principal pattern avoidance class is 
either sum or difference closed since if A = Av(n) then A is sum-closed 
(difference-closed) if 7r has no representation of the form 9 © r {6 r) 
with both 9 and r non-empty However, no permutation can be written 
both as a proper sum and as a proper difference. 

Theorem 2. Let A be an infinite and proper pattern avoidance class 
which is either sum-closed or difference-closed. There exists a constant 



E(a) 
( n )s 



n \ 




— 2e v / 3n 
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1 < ca < 00 suc h that 



(1) 



ri- 



lim 



■00 



E^(n n ) 

v 7 ™ 



Proof. Without loss of generality assume that A is sum-closed (the set 
of reversals of permutations in a difference closed class is sum-closed). 
The proof is a direct modification of Hammersley's for L n . We repeat a 
sketch of the details here. A more complete exposition can be found in 
|H]. Take a Poisson point process in (0, 00) x (0, 00) with unit density 
per unit area. Let A k be the length of the longest A subsequence in the 
k x k box with lower left corner at (0, 0). Then because A is ©-closed: 



where T k shifts the origin to (k, k) and preserves measure. The sub- 
additive ergodic theorem applies to — Ak and implies that A^/k con- 
verges almost surely to some value in [0, 00]. Conditioned on the 
number of points in the k x k square being n, Ak is distributed as 
LaO^-u)- For large k, n is approximately k 2 . Every sum-closed class 
contains all increasing permutations which establishes the lower bound 
claimed. Furthermore, one consequence of the lemma above is that 
limsup^^ La(X) I \fn < 00 so the full result follows. □ 

Reasons for the obscure choice of limiting constant in equation ^ will 
be given in the next section. We now present a modification of the 
arguments in [§] to show that the expectation of L^(Il n ) is tightly 
concentrated around its mean. 

Theorem 3. Let A be a proper pattern class. For a > 1/3 and (3 < 
min(o;, 3a — 1) 



Proof. With the probability bound supplied by the lemma above in 
place the remainder of the proof for tight concentration is virtually 
identical to that of 9J. At some points in that argument various con- 
stants appear (for example, 6 whose significance is that it dominates 
2e). Owing to the difference in the probability bounds, these constants 
must all be multiplied by yfs. Otherwise the argument is completely 
unaffected, since it never actually makes use of any properties of in- 
creasing sequences. □ 



Ak+m > A k + A m o T k 



(2) 



Pr (|L^(n n ) - EL4(n n )| > n a ) < exp(-r/). 



We remark at this point that the reason we have not been able to 
give the tighter concentration bounds supplied by |H| is that the argu- 
ments of that paper do use special properties of increasing sequences. 
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Specifically, they make use of a decomposition of the unit square into 
subsquares and the fact that if an increasing subsequence contains an 
element from a particular subsquare, then this generally rules out ele- 
ments in many other subsquares. However, the form of (J2J) is borrowed 
from [S| where the observation is made that this form follows directly 
from the proof provided in [5]. 



The Marcus- Tardos Stanley- Wilf theorem was a key ingredient in the 
proof of Theorems El and EJ We will now present observations which 
provide some evidence for a connection between the constants ca and 



A superadditivity argument due to Arratia [3] establishes that the right 
hand side is an actual limit for any principal pattern avoidance class, 
and more generally for any sum or difference-closed pattern avoidance 
class. 

For convenience we adopt a similar definition for C4 that applies even 
if we cannot be sure that the limit given in equation Q exists. 



Certainly Lemma Q implies that for an infinite and proper pattern class 
A, 1 < < 65.4. 

Conjecture 1. For any proper pattern avoidance class A, the limits 
superior definining C4 and s A are in fact limits, and C4 = s^. 

This conjecture provides the reason for the otherwise obscure choice 
for the form of the right hand side of equation The similarity of 
the arguments required to prove that both limits superior above are 
actually limits in the sum or difference-closed case is our second piece 
of (weak) evidence for the conjecture (the first is that it applies to X!) 
The remainder of the paper is devoted to an investigation of the rather 
fragmentary evidence in support of Conjecture H This evidence takes 
two forms: a series of propositions that establish the equality claimed 
by it holds for certain classes and is preserved by certain constructions, 
and some limited experimental data on two specific classes. We begin 
with the more theoretical evidence. 



3. Meaning of the constant 



s A . 



Recall the definition of sx- 



s_4 = limsup \S n R A\ 



l/n 
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It will be convenient to use the expression "A satisfies Conjecture [T]' 
as an abbreviation for the more accurate "the limits superior defining 
c_4 and s_a are in fact limits, and C4 = s_4". We will also consider 
occasionally the weak form of the above where only the equality of 
C4 and s.4 is asserted and not the fact that these constants can be 
computed as limits rather than as limits superior. 

Proposition 4. For each positive integer k, the classes 

Av(Jfe(Jfe-l)(A;-2)---21) and Av(123---Jfe) 

satisfy Conjecture^ 

Proof. By symmetry it suffices to consider T(k) = Av(k(k — l)(k — 
2)---21). Greene ^T] proved that the length of the longest T{k)- 
subsequence of an arbitrary permutation 7r is the sum of the lengths 
of the first k — 1 rows of the tableaux produced from ir through the 
Robinson-Schensted-Knuth algorithm. From the detailed asymptotics 
for the shapes of such tableaux as can be found in [7j we deduce that 

lim ELj(fc ) (n " } = 2(k - 1) 

and therefore c%(k) = (k — l) 2 - On the other hand results of Regev jTH] 
(and see also jTU]) demonstrate that sj(k) = (k-1) 2 . □ 

We now consider four constructions which when applied to pattern 
avoidance classes A and B define a pattern avoidance class C D A U B. 
The first of these is the union construction itself, the second is direct 
sum (which we have seen above), the third is called juxtaposition, and 
the fourth merge. 

The juxtaposition Juxt(^4, B) of two pattern avoidance classes consists 
of the set of all permutations of the form a(3 where the pattern of a 
lies in A and that of (3 lies in B. The merge Merge (A, B) consists 
of the set of all permutations 7i"i7r2 ■ • -7T n such that for some subset 
I C {1,2, ... ,n} the pattern of the subpermutation of rr consisting of 
elements whose indices lie in / belongs to A, and the pattern of the 
remaining elements lies in B. 

Proposition 5. Let A and B be two pattern avoidance classes which 
satisfy Conjecture Then their union, direct sum and juxtaposition 
also satisfy Conjecture Ql If, additionally, An B is a finite class then 
their merge also satisfies Conjecture^ 
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Proof. Suppose that A and B are pattern avoidance classes which sat- 
isfy Conjecture [U and let a k and b k denote the number of permutations 
of length k in A and B respectively. 

Let U = A U B and let u k denote the number of permutations of length 
k in U. Then: 



So U satisfies Conjecture H The argument for direct sum is almost 
equally simple based on the observation that the number of permuta- 
tions of length n that belong to the direct sum is at least equal to the 
maximum of a k b n _ k for < k < n and at most n times this quantity. 
This establishes that the Stanley- Wilf limit for the direct sum is the 
maximum s A and sg. The same result follows for c a ^b using Theorems 
ElandU 

Now let J = Juxt(^4, B) and let j k denote the number of permutations 
of length k in J . Then: 



since each permutation in J of length n has at least one and at most 
n+ 1 representations as a juxtaposition of a permutation in A and one 
in B. Observing the similarity of the middle expression to the binomial 
expansion of (s^ + Ss) n one obtains by taking nth roots that 



On the other hand the arguments of Steele in (201 applied to juxta- 
positions of arbitrary pattern avoidance classes rather than just the 
increasing and decreasing class together with the theorems above also 
yield: 

Cj = c A + C B . 
So Conjecture H] holds for Juxt(^4, B). 

Finally, suppose that A D B is finite, and let s be the maximum length 
of a permutation in A D B. Let M. = Merge (A, B) and let jk denote 
the number of permutations of length k in J . Consider first cm- Given 
any permutation ir, the merge of any of its longest ^.-sequences with 



max(a fe , b k ) < u k < a k + b k . 
It follows immediately that su = max(s^, sg). 
Using Theorems 121 and El it follows that: 



c u = max(cA,c s ). 
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any of its longest £>-sequences is an .M-sequence in tx. Conversely, any 
.M-sequence is a merge of an .4-sequence and a £>-sequence. So 



L A (ir) + L b (tx) - s < L M (ir) < L A (n) + L B ( 



7T . 



Since s is fixed, on taking expectation dividing by 2y/n and taking a 
limit as n — > oo we obtain 



\/c~m = \/ca + \fc~B- 

So C M = (y/CA + x/CbY- 

To form the merge of a permutation of length k and one of length n — k 
we choose k positions for the first permutation and also k values. So: 

n / \ 2 

v I n\ 

m n < 



E u akK < 

h—n V / 



k=0 

If 7i G M. is represented in two different ways as the merge of a per- 
mutation in A and one in E then the number of positions in which the 
two merges disagree (in the sense that one is representing the value in 
that position as part of an ^4-permutation and the other as part of a 
£>-permutation) is at most 2s. This is because if we consider positions 
represented in the first merge as coming from an ^4-permutation and 
in the second as coming from a £>-permutation, the pattern occurring 
at these positions belongs to AC)B (and likewise for the opposite case). 
So, labeling positions in a merge by type, the merges representing ir 
correspond to a set of diameter at most 2s in the Hamming metric 
on the cube {0,1 } n . Kleitman ^3] proved that the maximum num- 
ber of points in such a set is attained in a ball of radius s, that is 
v{n,s) = J2 S t =o 0- So 

n / \ 2 

m n <y^( j a k b n _ k < v(n, s)m n . 
Since s is fixed, v (n, s) 1//n ^lasn^ oo. Therefore 
s M = lim S~] I ) a k b n - k 

— V^y 

Considering the similarity of the square root of each term of the sum 
in the expression above to a term in the expansion of (^fsX + ^/sb) 11 
we obtain: 

\2 



SM = W S -A + V S Bj 

and therefore M. satisfies Conjecture [T] □ 
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The results of the preceding proposition apply in part to the weaker 
version of Conjecture Q which only asserts the equality of two limits 
superior. Namely, following the details of the proof we see that if A and 
B satisfy the weaker form then so do their union and direct sum, while 
if one satisfies the stronger form and the other the weaker form then 
their juxtaposition and merge (subject to finite intersection) also satisfy 
the weaker form. Note also that the condition that A H B be finite is 
equivalent to asserting that for some k and k', k(k — 1) • • • 321 ^ A(lB 
and 123 • • • k' AC\B. The "interesting" cases are when neither A nor 
B is finite, say A C Av(k(k - 1) • • • 321) and B C Av(123 • • -k'). 

Let A be any pattern avoidance class and define Rot (A) to be the set 
of permutations obtained by taking all the cyclic rotations of elements 
of A. It is easily verified that Rot (.A) is also a pattern avoidance class. 

Proposition 6. If A satisfies Conjecture^ then so does Rot (A). 

Proof. Let A be a pattern avoidance class satisfying Conjecture Hand 
let 71 = Rot (A). Since 1Z D A and the number of elements of 1Z of 
length n is at most n times the number of elements of A of length n, 
sn = sa- 

However, the definition of c-n and Theorem |3] also imply that c-n = 
C4. To see this observe that the length of any LTZS sequence in a 
permutation 7r is the maximum of the lengths of an LAS sequence in 
the cyclic rotations of tt. The n! permutations in S n are partitioned 
into (n — 1)! classes of n elements under cyclic rotation, and the mean 
length of an LTZS sequence is the mean of the maximum lengths of the 
LAS sequences in each of these equivalence classes. Thus the mean 
length of an LTZS sequence is not greater than M n , the mean length of 
an LAS sequence over the (n — 1)! permutations whose LAS sequences 
are longest. Choose 1/3 < a < 1/2 and < (3 < 3a — 1. Then, using 
Theorem |SJ 

M n < (EL^(n n ) + n a ) +n 2 exp(-n f3 ). 

The final term arises from the upper bound of n for the length of the 
longest A subsequence. In particular: 

hm —= < 2-V/C4. 

rwoo y/fl 

This gives c-n < C4 and the reverse inequality is of course trivial. Thus 
1Z satisfies Conjecture [T]as claimed. □ 

Again, the "weak" form of this proposition is valid. 



LONGEST PATTERN AVOIDING SUBSEQUENCES 



11 



4. Experimental results 

The results of the previous section provide some evidence in favour 
of Conjecture ^ or more conservatively, provide evidence that for a 
fairly wide set of pattern avoidance classes A, C4 = s^. However, we 
have no significant evidence in favour of this conjecture based on an 
exact computation of C4 for any classes other than those which can be 
produced from the classes Av(k(k — 1) • • • 321) or Av(123 • • • (m — l)m) 
by the constructions described in the previous section. 

In this section we consider two specific pattern avoidance classes: 

• The layered permutations, £ = Av(231,312), consisting of all 
permutations of the form D\ © D 2 © • • • © where each of 
Di through is a descending permutation. The number of 
permutations of length n in £ is 2 n ~ 1 and so sc = 2. 

• The subclass £{2) = Av(231, 312, 321) of £ formed by requiring 
that each Di contain at most two elements. The number of 
permutations of length n in £(2) is equal to the nth Fibonacci 
number, so s C (2) — (1 + \/5)/2. Kaiser and Klazar [THj proved 
that £(2) is the smallest pattern avoidance class whose Stanley- 
Wilf limit is strictly greater than 1. 

Results in |T] give dynamic programming algorithms for solving the 
longest subsequence problem for both £ and £{2) whose complexity is 
0(n 2 logn) where n is the length of the input permutation. We have 
been able to improve the latter algorithm, based on a tableau style 
method to result in a complexity of O(nlogn). Both algorithms were 
implemented and a long period random number generator was used to 
provide experimental data concerning the values cc and cn2)- 

For £{2) we present data based on permutations of length 2 k x 10 4 for 
< k < 7. For each value of k, 1000 random permutations of that 
length were generated and the length of the longest £{2) subsequences 
was computed. Table [T] shows the mean, sample standard deviation, 
and resulting estimates of cc(2) based on these simulations. We would 
be forced to classify a person who believed in the truth of Conjecture [T] 
based on this data for £(2) as an optimist. If the estimates are indeed 
converging to Sc(2) then they are not yet within 6% of their final limit 
at n = 128 x 10 4 . By contrast, for this value of n the estimate for cj 
(whose actual value is 1) is approximately 0.985. 

Because of the slower running time and increased space requirements 
required by the algorithm for finding longest layered subsequences data 
for £ is based on permutations of length 2 k x 10 2 for < k < 7. As 
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Length 



1 x 10 4 

2 x 10 4 
4 x 10 4 
8 x 10 4 
16 x 10 4 
32 x 10 4 
64 x 10 4 
128 x 10 4 



a 



239.3 4.5 1.431 
340.7 5.2 1.451 
484.7 6.1 1.468 

688.4 6.4 1.481 
978.1 7.1 1.495 
1386.8 8.3 1.503 
1965.3 9.3 1.510 
2785.3 10.2 1.515 

Table 1 . Summary data for the mean, /i, and standard 
deviation a of the length of the longest £(2) subsequences 
of random permutations together with corresponding es- 
timates of C£( 2 ). 



C C(2) 



Length 




a 


~ Cc 


1 x 10 2 


23.8 


1.8 


1.418 


2 x 10 2 


34.8 


2.2 


1.517 


4 x 10 2 


50.6 


2.5 


1.602 


8 x 10 2 


73.4 


3.0 


1.682 


16 x 10 2 


105.2 


3.3 


1.730 


32 x 10 2 


150.7 


4.0 


1.774 


64 x 10 2 


215.9 


4.4 


1.821 


128 x 10 2 


307.5 


4.9 


1.847 



Table 2. Summary data for the mean, /i, and standard 
deviation a of the length of the longest C subsequences 
of random permutations together with corresponding es- 
timates of Cc- 



for £(2), 1000 random permutations of each length were analysed and 
the results are presented in Table 121 The data for this class do not 
require as much optimism as the C{2) data to be viewed as support for 
Conjecture 1. 



5. Conclusions 

We have illustrated that, broadly speaking, the statistic on S n 
has the same general qualities as the more well known C% statistic. 
Certainly a great deal more work is required before one could begin 
to appreciate the former statistic at the level of detail that is known 
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concerning the latter. Not surprisingly, it appears that there is a con- 
nection between £4 and the growth rate of the pattern avoidance class 
A. Conjecture [T] proposes a precise form for this connection and the 
results of Section El establish a relatively wide collection of classes for 
which the Conjecture^ holds. One extension to these results concerns 
classes A for which = 1. These are (essentially) classified in |T3] 
and it is only a matter of somewhat tedious routine to confirm that in 
all these cases C4 = 1 also. 

However, the experimental results of Section 0] can be viewed as provid- 
ing evidence against Conjecture [TJ In particular the class C(2) seems 
a likely contender as a possible counterexample. The structure of the 
permutations in this class is sufficiently simple that there may be some 
chance of carrying out a more detailed analysis of £c(2) with a view to 
explicitly calculating, or providing bounds for, c^y 

Notable by its omission from either Section |3] or Section 0] is the class 
Av(312). This class has Stanley- Wilf limit 4 (as does every class defined 
by avoiding a single three element pattern since all such classes are 
enumerated by the Catalan numbers). A polynomial time algorithm 
for the longest subsequence problem based on this class is given in pQ 
but its complexity on permutations of length n is 0(n 5 ) which makes it 
impractical for experiments of the size required to produce even vaguely 
convincing evidence. The goal of producing such evidence would seem 
to require finding, even on an ad hoc basis some collection of classes for 
which the longest subsequence problem can be solved algorithmically 
in reasonable time (basically, at worst quadratic) and/or developing 
better algorithms for classes such as Av(312). 
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